A Catch 22 of 3D Data Sustainability: Lessons in 3D Archaeological Data Management & Accessibility

Archaeologists can now collect an inordinate amount of 3D data. But are these 3D data sustainable? Are they being managed to make them accessible? The MayaArch3D Project researched and addressed these questions by applying best practices to build four prototype tools to store, manage, visualize, and analyze multiresolution, geo-referenced 3D models in a web-based environment. While the technical aspects of these tools have been published, this position paper addresses a catch 22 that we, as archaeologists, encounter in the field of 3D archaeology – one that formed the initial impetus for the MayaArch3D Project: that is, while the quantity of 3D archaeological data is increasing, these data are not usually accessible. By researching and addressing 3D data integration and accessibility, we learned many lessons that group around four main issues: sensitivity/security, web-based dissemination, conveying uncertainty, and data storage/reuse/peer review. These are significant current challenges to making 3D archaeological data sustainable. 38 digitalcommons.unl.edu Lessons in 3D archaeolog ical data management & access i b i l i ty 39 While efforts for 3D archaeological data sustainability are growing in the U.S. (e.g., www.cyark.org/ ), European institutions and organizations particularly have been working in the area of 3D data management toward establishing best practices and developing infrastructure for 3D data management in archaeology and cultural heritage for the past decade, (Remondino and Campana, 2014; Cignoni and Scopigno, 2008; Fresa et al., 2015; Lyons, 2016; Stylianidis and Remondino, 2016; Taylor and Gibson, 2016; Vecchio et al., 2015; von Schwerin et al., 2009, 2011; www.dc-net.org/getFile.php?id=467 ; www.dc-net.org/getFile. php?id=450 ). Efforts include CARARE ( www.carare.eu/ ) and 3D-COFORM (www.3d-coform.eu/), which built tools, infrastructure, and workflows, and were essential in establishing a community of experts. Europeana ( www.europeana.eu/portal/ ) integrates workflows and software developed through CARARE to promote re-use of cultural heritage data. The 3D-Icons Project ( http://3dicons-project.eu/ ) built on these earlier efforts, focusing on the documentation and distribution in 3D data for UNESCO World Heritage Sites (Corns et al., 2015). Current projects including VCC-3D (www.vcc-3d.com/) and ARIADNE ( www.ariadne-infrastructure.eu/ ) concentrate on providing centralized expertise and centralized infrastructure to the community, whereas the INCEPTION project (www.inception-project.eu/) continues to innovate in the development and implementation of new technologies, and focuses on 3D semantic modeling. The Archaeological Data Service (ADS), located in the UK, provides a guide to good practice for archaeological data that includes close-range photogrammetry, laser scanning, and virtual reality with information on archiving these data ( http://guides. archaeologydataservice.ac.uk/g2gp/Main ). Applying these good practices, ADS recently developed a web-based 3D Viewer for the management and analysis of archaeological 3D data. The ADS 3D Viewer integrates the 3D Heritage Online Presenter (3DHOP), a software package for the web-based visualization of 3D geometries, with the infrastructure of the Archaeology Data Service (ADS) repository (Galeazzi et al., 2016; Potenziani et al., 2015) to make individual 3D models with associated archaeological data more accessible. While the development of standards and best practices for 3D content is an essential component of data management, in this paper we address data sustainability with an end goal of reuse. Our key point is that to achieve 3D data sustainability, we not only need to guard against obsolete file types and disappearing data, but more work is required to make the data reusable. We understand that best practices and standards underlie data reuse, but continual management of the data is also required – this will allow for the adjustment of data to be used for multiple purposes, and as standards and best practices change. If we facilitate 3D data reuse for multiple purposes, this will lead to greater data sustainability because institutions have an impetus to sustain data that are being used and reused. To make data reusable, it must be accessible. The common definition of data accessibility refers to a user’s ability to access or retrieve data stored within a database or other repository. But what does it actually mean to be able to access data? Does this mean users can view data online, or must they be able to retrieve, move, and manipulate data? We contend that a key component to making data truly accessible is providing the ability to do something with these data, that is, users must be able to reuse them in some way to further research. In this regard, the concept of linked data, i.e., interrelated datasets on the Web, which necessitates “standard [data] formats and reachable and manageable Semantic Web tools” provides a framework for “defining” data accessibility ( www.w3.org/standards/semanticweb/data ). Open Context—a web-based open access publishing platform—uses open linked data to provide data management as well as publishing for archaeological data that facilitates not only data access but data reuse ( https://opencontext.org/about/ ; Arbuckle et al., 2014; Buccellati and Kansa, 2016). However, currently they do not offer a platform for visualizing and querying 3D data. The task of making 3D archaeological data reusable is difficult not only because of limited data standards, heavy datasets, and rapidly changing technologies, but also because as archaeologists we must seriously consider to whom we can “ethically” make these data accessible (Vitelli and Colwell-Chanthaphonh, 2006). Furthermore, the difference between 3D surveying and 3D modeling, which underscores the difference between data acquisition vs. data creation, adds a layer of complexity, fuzziness, and interminability to the task. 1. The problem of “Infinite” 3D data 3D surveying is data collection. It is the raw x, y, z data we collect, for example with airborne LIDAR, terrestrial laser scanning, photogrammetry as well as theodolites. Data resolution ranges from microns to meters depending on technology and acquisition choices; the higher the resolution, the more 3D points, and thus the greater the storage requirements. To sustain these data, they must be stored, and data storage is expensive. Additionally, questions of data format arise—many data are collected in proprietary formats. What happens if we store only the proprietary format and the software to visualize and manipulate them disappears? Are inaccessible data that are stored on hard drives, servers, or the cloud truly sustained? No. This is why data management—not only data storage—is crucial to data sustainability. To further complicate this problem, much 3D archaeological data comes from 3D modeling. 3D modeling is what we do with these data— in other words it is another layer of data creation; for example, we convert point clouds into polygonal meshes requiring decisions, interpretations, and transformations of the raw data. Along these lines, we reference two model types: (1) base models derived from laser scanning, photogrammetry, etc., that is, 3D technologies that collect data on extant archaeological features such as standing architecture, and (2) 3D reconstructions that hypothesize how a building or object might have looked in the past based on integrating sources ranging from architectural plans to laser scans to excavation data. With both types of modeling, interpretive decisions are made and these decisions (paradata) are critical to the 3D models because they explain the choices modelers have made, for example, as they decimate meshes of base models for online use or add a third story to a building (Bentkowska-Kafel et al., 2016; Lyons, 2016; von Schwerin et al., 2011). The take-away points are three-fold. First, a 3D model is an interpretation and several reconstructions might exist for an individual building—which reconstruction do we keep? Do we keep all of them? Second, modeling involves choices (paradata) based on the input of data sources. How do we sustain the original data sources and paradata as well as the 3D data? Third, software is continuously updated and unfortunately newer versions are not always compatible with previous versions; thus it is always necessary to record the software version used to create and visualize 3D data. Which version of a 3D model do we preserve? Do we preserve all of them? To provide an illustration, a quick search for 3D file formats on Wikipedia ( https://en.wikipedia.org/wiki/List_of_file_formats ) returns over seventy-four file formats for 3D models (graphic), and with the growing popularity of virtual and augmented reality this number is increasing. Among these many file formats, a few serve as “standards” for 3D archaeological data; however, these file types have pros and cons based on storage, visualization, and analytical needs. For example, the STL (STereoLithography) format is often used for 3D printing because it stores closed, solid (“watertight”) models that can be printed in slices, but a major drawback is that STL files have large sizes and do not store textures or other material properties. Moreover, in archaeology we often do not have “watertight” models – particularly for 3D landscape data. And, what about textures or other material properties? Is it enough to only store geometry when materials are critical for archaeological analysis? Thus, while STL might be an ideal format for 3D printing, it only 40 R ichards-R i s setto & von Schwer in in Archaeology and Cultural Her i tage 6 (2017 ) serves a partial solution to the broad needs of 3D archaeological data sustainability and accessibility. Typically used for terrestrial laser scanning data is PLY—an ASCII polygon or binary format with no compression that stores both geometry and textures. However, for archaeology additional material properties are often critical, and in this case the Wavefront OBJ format, a simple data format that stores 3D geometry with texture coordinates along with a Material File(s) (.MTL) to define specific material properties such as shininess, is preferable. The common denominator among these three files types, STL, PLY, and OBJ, is that they can all be stored as ASCII and this is critical for sustainability as it is simply text that can be read without special software. In the end, the number of ways we can transform our data is almost infinitesimal—so what do we preserve? How do we sustain the infinitesimal? To discuss some of these issues in more detail, we use as an example our experiences as members of the MayaArch3D Project—a project that developed a 3DWebGIS to bring together GIS and 3D for studies of ancient Maya architecture and landscapes ( www.mayaarch3d.org ). 2. The MayaArch3D project The MayaArch3D project developed an open source, 3D WebGIS for the documentation, visualization, and analysis of complex archaeological sites (Reindel, 2014, von Schwerin, 2013, von Schwerin et al., 2016a, 2016b; Auer et al., 2014). This system has four prototype tools that can store, manage, query, visualize, and analyze 3D data of different formats and resolutions. Project partners were the Commission for the Archaeology of Non-European Cultures (KAAK) of the German Archaeological Institute (DAI), Department of Geoinformatics at the University of Heidelberg, Department of Anthropology and Center for Research in the Digital Humanities (CDRH) at the University of Nebraska-Lincoln, the Honduran Institute of Anthropology and History (IHAH), and the 3D Optical Metrology Unit of the Bruno Kessler Foundation (FBK). The technical aspects of this system have already been published (Auer et al., 2014; Billen et al., 2013; Loos et al., 2013, Lyons, 2016, Reindel et al., 2016, von Schwerin et al., 2015, von Schwerin et al., 2016a, 2016b). In this section we give a brief overview of the system tools; however, because technology so rapidly changes, we focus on data management issues underlying 3D archaeological data accessibility that ultimately impact sustainability. The case study is the archaeological site of Copan, which today is a UNESCO World Heritage site located in Honduras (Fig. 1). The site’s occupation dates back to at least 1800 BCE (Hall and Viel, 2004) and from the fifth to ninth centuries CE, Copan was a prominent Maya Kingdom (Fash, 2001). Copan is an ideal site for development and testing of 3DWebGIS because it has a long history of excavation and research providing large and diverse datasets from many researchers. In addition to archival and published data, the MayaArch3D Project has also collected close-range photogrammetric data, terrestrial laser scans, airborne lidar, and created CAD and Geographic Information Systems (GIS) data including, for example, .3ds, .obj, collada, shapefiles, and Digital Elevation Models (Remondino et al., 2009; Reindel et al., 2016; Richards-Rissetto, 2010; Richards-Rissetto, 2013; Richards-Rissetto and Landau, 2014; Richards-Rissetto, 2017; von Schwerin et al., 2013, 2016a, 2016b). We have used these data for the development of four prototype web-based tools: 2D Geobrowser, 3D Single Object Viewer, 3D Scene Viewer, and Virtual Panoramic Tour. We emphasize that these tools are prototypes, and that during tool development, we learned several lessons in managing 3D archaeological data with the intent to make it accessible. We believe these lessons learned can assist others in more effective planning for sustainable data management as they design archaeological projects involving 3D data (see also von Schwerin et al., 2016a, 2016b). 3. MayaArch3D prototype tools